System and method for tracking advertiser return on investment using set top box data

ABSTRACT

A processing device receives set-top box data, sales data, and advertising data. The processing device associates set-top boxes front the set-top box data with viewers and calculates advertising exposure events for advertisements. The processing device further calculates, for each viewer, an advertising weight as a function of number of exposures of the advertisement as applied to the viewer based on the advertising exposure events. The processing device further calculates, for each viewer, a score representing a degree-of-targetedness between an advertisement campaign and the viewer. The processing device manages the advertisement campaign based on the advertising weight and the degree-of-targetedness.

RELATED APPLICATIONS

This patent application claims the benefit under 35 U.S.C. § 119(e) ofU.S. Provisional Application No. 61/709889, filed Oct. 4, 2012, which isherein incorporated by reference,

TECHNICAL FIELD

Embodiments of the present disclosure relate to the field of mediaadvertising and, more particularly, to tracking and managing advertisingcampaigns.

BACKGROUND

Tracking return on investment (ROI) from television (TV) is an unsolvedproblem for advertising. Some of these attempts are set forth below.

A. IPTV—Many commentators have written that efforts such as internetprotocol enabled television (IPTV) will eventually enable TV conversionsto be tracked via conversion tracking pixels similar to those in placetoday throughout the web. IPTVs obtain their TV content from theinternet and use hypertext transport protocol (HTTP) for requestingcontent. However, there are many technical challenges before trackingconversions using IPTVs becomes a reality. Today, only about 8% of US TVhouseholds have IP enabled TV. Attempts to introduce IPTVs such asGoogle® TV and Apple® TV have met with only lukewarm interest. Even ifweb-like conversion tracking becomes possible using TV, it still won'tcapture all of the activity such as brand recognition leading to delayedconversions, and purchasing at retail stores.

B. RFI Systems—Some companies have experimented with methods forenabling existing TVs to be able to support a direct “purchase” from“the consumer's lounge room” using present-day Set-top box systems andremote controls. Some system providers have developed an on-screen “bug”that appears at the bottom of the screen, and asks the consumer if theywould want more information or a coupon. The consumer can click on theirremote control to accept. Although promising, adoption of remote controlRFI systems is constrained by lack of hardware support and standards.These systems also have the same disadvantages of IPTV, in being unableto track delayed conversions.

C. Panels—One of the most common fallbacks in the TV arena—when facedwith difficult-to-measure effects—is to use volunteer, paid panels tofind out what people do after they view advertisements. There areseveral companies that use panels to try to track TV exposures to sales.One advantage of this method is that it makes real-time trackingpossible. However, in all cases, the small size of the panel (e.g.,25,000 people for some panels) presents formidable challenges forextrapolation and difficulty finding enough transactions to reliablymeasure sales. Another problem with the panel approach is the cost ofmaintaining the panels.

D. Mix Models—If data from previous campaigns has been collected, thenit may be possible to regress the historical marketing channel activity(e.g., impressions bought on TV ads, radio ads, web ads, print, etc.)against future sales. Unfortunately, such an approach offers no help ifthe relationships change in the future. Moreover, such an approach doesnot provide real time tracking. In addition, historical factors arerarely orthogonal—for example, retailers often execute coordinatedadvertising across multiple channels correlated in time on purpose inorder to exploit seasonal events. This can lead to a historical factorsmatrix that aliases interactions and even main effects. Even if thereare observations in which all main effects vary orthogonally, inpractice there may be too few cases for estimation.

E. Market Tests—Market Tests overcome the problems of mix models bycreating orthogonal experimental designs to study the phenomena underquestion. TV is run in some geographic areas and not others, and salesthen compared between the two. Market tests rely on local areas tocompare treatments to controls. One problem typical to market tests istheir inability to be used during a national campaign. Once a nationaltelevision ad campaign is under way, there are no longer any controlsthat aren't receiving the TV signal of the ad campaign. This causesadditional problems—for example, a market test might be executedflawlessly in April, and then a national campaign starts up in May.However, some external event is now in play during May, and the findingscompiled meticulously during April are no longer valid. This is aproblem of the market test being a “research study” that becomes “stale”as soon as the national campaign is started. Thus, market tests alsofail to provide real time tracking.

None of the above methods or techniques provide practical methods toeffectively track the effects of TV advertising in real-time in anational campaign. The lack of conversion tracking on TV has arguablyled to a proliferation of untargeted, irrelevant ads. Solving the TVconversion tracking problem could be of great significance forcomputational advertising.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will be understood more fully from the detaileddescription given below and from the accompanying drawings of variousembodiments of the present invention, which, however, should not betaken to limit the present invention to the specific embodiments, butare for explanation and understanding only.

FIG. 1 illustrates four different television (TV) providers to ahousehold, as well as return path data for each.

FIG. 2 is a block diagram of a system architecture in which embodimentsof the present invention described herein may operate.

FIG. 3 illustrates a number of hours between tune events byset-top-boxes.

FIG. 4 shows the average time between tune events.

FIG. 5 illustrates a percentage of set-top box viewers who saw 1, 2, 3or more ads.

FIGS. 6-7 illustrate advertisement weight verses conversion probability,in accordance with one embodiment.

FIGS. 8-9 show the relationship between demographic match andprobability of conversion, in accordance with one embodiment.

FIG. 10 illustrates a graph showing combined ad-weight and targeting(targetedness) verses probability of conversion, in accordance with oneembodiment.

FIG. 11 is a flow diagram illustrating a method for real time trackingof an advertisement campaign, according to one embodiment.

FIG. 12 is a flow diagram illustrating a method for real time trackingof an advertisement campaign, according to another embodiment.

FIG. 13 illustrates a diagrammatic representation of a machine in theexemplary form of a computer system.

DETAILED DESCRIPTION

It is generally difficult to track the effects of TV advertising (e.g.,track whether a TV advertisement resulted in a purchase in a store oronline). Providing systems and methods to track the effects of TVadvertising in real time may allow more complete cost and revenuemetrics to be maintained for the TV advertising, which may in turn allowfor better targeting, optimization, and control of advertisementsbecause of visibility into their performance.

The system and method described in embodiments exploit an important newsource of data to perform conversion tracking. Set-top box data makes itpossible to measure advertisement (ad) delivery to specific households.Each household receives a certain exposure of ads, and then eitherconverts or does not convert. Analyzing the ad weight load andtargeting, it is possible to identify how consumers are responding totelevision (TV) advertising. In addition, it is also possible to simplytrack how well an advertising campaign is doing at getting ads in frontof buyers. Embodiments provide advantages including being able toutilize large populations, and being implementable using existingtelevision hardware.

In embodiments, set-top box data is received and processed. The set-topbox data is then combined with three other data sources to be able tosee ad exposures and conversions.

The following description sets forth numerous specific details such asexamples of specific systems, components, methods, and so forth, inorder to provide a good understanding of several embodiments of thepresent invention. It will be apparent to one skilled in the art,however, that at least some embodiments of the present invention may bepracticed without these specific details. In other instances, well-knowncomponents or methods are not described in detail or are presented insimple block diagram format in order to avoid unnecessarily obscuringthe present invention. Thus, the specific details set forth are merelyexemplary. Particular implementations may vary from these exemplarydetails and still be contemplated to be within the scope of the presentinvention.

FIG. 1 illustrates four different television (TV) providers to ahousehold, as well as return path data for each. The four different TVproviders include a satellite TV provider, a cable TV provider, aninternet protocol TV (IPTV) provider and an over-the-air TV provider.The satellite TV provider includes a broadcast center 115 that directsTV signals to a direct broadcast satellite 120. The direct broadcastsatellite 120 then directs a TV signal 122 to a dish receiver 125 of ahousehold. The TV signal is transmitted from the dish receiver 125 to aset-top box 130, which translates the TV signal 122 into a format thatcan be displayed on a TV. The set-top box 130 may receive scheduleupdates, pay-per-view (PPV) data, video on demand (VOD) data and/orsystem updates from a satellite front end server 142 on a secondary path138. The secondary path 138 may be a dial up phone connection, atransmission control protocol/internet protocol (TCP/IP) connection, orother connection. The secondary path 138 may additionally act as areturn path that the set-top box 130 may use to provide data to thefront end server 142. The data may include stored set-top box data,credit card data, video rental data, and so on.

The cable TV provider includes a head end multiple system operator (MSO)140 that provides a TV signal and/or updates to the set-top box 130 overa cable connection, which may be a coaxial cable connection in someimplementations. The set-top box 130 may additionally provide set topbox data to the head end 140 via a return path 152 on the cableconnection.

The internet protocol television (IPTV) provider includes an internetservice 165 that may connect to the set-top box 130 via a TCP/IPconnection. Via the TCP/IP connection, the internet service 165 mayprovide a TV signal and updates 155, and may receive set-top box data ina return path 160.

The over-the-air TV provider includes an over-the-air broadcast center145 that broadcasts an over the-air TV signal. An over the air antenna128 receives the broadcast signal and may route the over-the-airbroadcast signal to set-top box 130 or directly to a TV.

A single set-top box 130 may receive all of the above mentioned TVsignals. Alternatively, different set-top boxes may be configured toreceive a particular type of TV signal. For example, the cable providermay provide a set-top box for the cable TV signal, and the satellite TVprovider may provide a set-top box for the satellite TV signal.

Set-top boxes 130 are small devices that interpret a TV signal (e.g., asatellite or cable TV signal), and turn it into content that can bedisplayed by television sets. Set-top boxes 130 emerged due to therapidly growing variety of TV signals and services that occurred from1990 to 2010, the increase of cable and satellite subscription over thesame period, and the move to all digital broadcasting in the UnitedStates in 2009. Amongst other things. Set-top boxes (i) demodulate cablesignals, (ii) interpret High Definition TV (HDTV), (iii) support VideoOn Demand (VOD) and Pay per view (PPV), (iv) support DVR (digital videorecorder) functionality, and (v) Satellite, Cable, IPTV and Terrestrialantenna broadcast digital protocols.

Set-top boxes have quietly revolutionized the television industry. Since1996, set-top boxes have increased from about 79% of households to over91.5% in the United States. Many households now have a set-top box witha return path capability. The specific kind of return path varies withthe data connection; for example IPTV installations can send data backin real-time, where-as Direct Broadcast Satellite often utilizes thephone line to update the unit once per day. Usage data may be collectedat Head-end (MSO) 140, Front End Servers (Satellite) 142 and InternetService Provider receiver servers (DPTV) 165.

In one embodiment, advertisement platform 180 connects to the head end140, front end server 142 and/or IPTV server 165. Advertisement platform180 may receive set-top box data (e.g., set-top box event records) fromthese systems, and combines the set-top box data with sales data and/oradditional data to determine conversion probability for ads and todetermine a quantity of purchasers who are exposed to the ads.

FIG. 2 is a block diagram of a system architecture 200 in whichembodiments of the present invention described herein may operate. Thesystem architecture 200 enables an advertisement platform 215 (e.g., theLucid Commerce®—Fathom Platform®) to collect data relevant to anadvertising campaign, to set up experiments, to track an advertisingcampaign in real time, and to otherwise control an advertisementcampaign. An advertising campaign may be a national advertisingcampaign, a state wide advertising campaign, a city wide advertisingcampaign, an advertising campaign targeting specific zip codes, and soforth. The system architecture 200 includes an advertisement platform215 connected to platform consumers 205 and systems that provideadvertising data 210, audience data (also referred to as usage data)220, sales data 225, and guide data 264.

The advertisement platform 215 receives as input the advertising data210, audience data 220, sales data 225, and guide data 264. Theadvertising data 210 may include media plan data (e.g., data indicatingadvertisements to run, target conversions (e.g., number of sales),target audiences, a plan budget, and so forth), verification data 244(e.g., data confirming that advertisements were run), and traffickingdata 246 (e.g., data indicating what advertisements are shipped to whichTV stations). Preferably, all the advertising data 210 about what mediais being purchased, run, and trafficked to stations is collected andprovided to the advertisement platform 215 to ensure that them is anaccurate representation of the television media. This may includesetting up data feeds for the media plan data 242, verification data244, and trafficking data 246.

The sales data 225 (also referred to as conversion data) includes dataon sales of products and/or services that are being advertised. Salesdata 225 may include, for example, call center data 252, electroniccommerce (ecommerce) data 254 and order management data 256. Theadvertisement platform 215 may set up a data feed to one or more callcenters to receive accurate data about phone orders placed by the callcenters for the advertised products or services. Additionally, recurringdata feeds may be set up with the vendor or internal system of theadvertiser that records orders that come in from the advertiser'swebsite (ecommerce data 254). Recurring data feeds with the order vendoror internal system that physically handles the logistics of billingand/or fulfillment may also be established (order management data 256).This may be used for subsequent purchases such as subscriptions and forreturns, bad debt, etc. to accurately account for revenue. This data mayalso originate from one or more retail Point of Sale systems.

The advertising platform 215 may generate a record for every caller,web-converter, and ultimate purchaser of the advertised product orserver that gets reported via the sales data 225. The advertisingplatform 215 may append to each record the data attributes for thepurchasers in terms of demographics, psychographics, behavior, and soforth. Such demographic and other information may be provided by databureaus such as Experian®, Acxiom®, Claritas®, etc. In one embodiment,advertiser data 225 includes consumer information enrichment data 258that encompasses such demographics, behavior and psychographicsinformation.

Audience data 220 may include set-top box data 262 (e.g., set-top boxrecords) and/or viewer information enrichment data 266. The viewerinformation enrichment data 266 may be similar to the consumer infoenrichment data 258, but may be associated with viewers of televisionprogramming as opposed to consumers of goods and services. A feed ofsuch viewer data 266 may include demographic, psychographic, and/orbehavioral data.

A set top box event record comprises the following tuple: (DeviceID,EventID, DateTime, TimeZone), where DateTime is recorded by the localset-top box unit. DeviceID is a unique identifier for the set-top box,the EventID is a field that indicates a type of event, DateTimeindicates the date and time the event occurred, and TimeZone indicatesthe time zone in which the set-top box is located. Each of these tuplesrepresents a remote control key press or state change event. The set-topbox events that are used in one embodiment are the “tune” events inwhich the user navigates and selects a channel, and the on/off events,in which the user turns on or off the set-top-box. Set-top box eventsare filtered by data source combiner 278 in one embodiment to isolatethe tune events and on/off events.

Data source combiner 278 may determine session boundaries for userviewing sessions (also referred to as session events) based on thefiltered set-top box event records. In one embodiment, there are twomethods for identifying session boundaries, in a first method, set-topbox on/off events are used to identify a session boundary. Sometimesconsumers will switch set-top boxes off after viewing, and on to startviewing, and these provide a very convenient way of identifying sessionstarts and ends. In a second method, inactive periods are used toidentify a session boundary. If there are no detected remote control keypresses for a certain number of hours, we determine that a session hasended.

FIG. 3 illustrates a number of hours between tune events byset-top-boxes. Based on an analysis of the time between last key pressand off events for set-top box data, there is a spike in off eventsafter 4 hours of inactivity that may be due to most viewing beingprime-time followed by a “cliff” around 4 hours later which coincideswith the sleep period. Additionally, based on an analysis of the timebetween key press events of any kind, there are spikes every half hour,which seem to be related to users switching to another program afterviewing a previous program. There are also large spikes in tune eventsat 4 and 24 hours. The 24 hour spike suggests that people may bewatching TV at around the same time each day. Based on the suggestivespikes at 4 hours, in one embodiment 4 hours is used as the inactiveperiod setting, in another embodiment, 6 hours of inactivity is used forcreating a session boundary.

One of the most salient behaviors in set-top boxes are the large numberof station request events that occur in quick succession as users “surf”channels looking for a program to watch. FIG. 4 shows the average timebetween tune events. Over 70% of events are “tune surf” events. Tunesurf events may be filtered by ensuring that the time a viewer spent onstation needs to be at least 10% of a specified viewing period (e.g.,1.5 minutes of each 15 minute bucket) for a station viewing session tobe generated from the set-top box event records. In some instances auser may jump back and forth between two programs, in which case tuneevents for neither program would be filtered out.

Some set-top box records for people who leave their set-top boxes on allof the time should also be filtered out. Although very high viewinglevels could indicate bad data, as a group, heavy viewers are actuallythe highest converters in the data. Heavy viewers have been shown toconvert 7 times higher than the lowest hour viewing segment. If thesystem normalizes by impressions, on a per impression basis, the high TVconsumption group and low consumption group actually have similarconversions per impression. This suggests that if ads are displayed tohumans, they tend to convert. In order to filter out the most extremeviewing behavior, but leaving in place the converters, data sourcecombiner 278 may filter out the most extreme heavy viewers who view morethan 99% of the population.

The guide service data 264 may include the programming of what is goingto run on television and/or what was run on television on variouschannels at various times.

All of the feeds of the various types of data may be received and storedinto a feed repository 272 by the advertisement platform 215. All of theunderlying data may be put into production and all of the data feeds maybe loaded into an intermediate format for cleansing, addingidentifier's, etc.

The advertisement platform 215 may ingest all of the data from the datafeeds. Data source combiner 278 coordinates the data from the differentdata sources. In one embodiment, data source combiner 278 matches useridentifiers in records from the different data sources (e.g., salesrecords and set-top box records), and combines these records with thematching user identifiers into a single combined record. Additionally,guide data and/or advertisement data may be combined with records fromthe set-top box data based on time and channel information. Combinedrecords may be used to determine conversion rates, conversionprobability, a number of converters who were exposed to ads from theadvertising campaign, and so on.

The data source combiner 278 joins the set-top-box data 262 toelectronic programming guide (EPG) data 264 to identify the programsthat were running during each of these determined viewing sessions oneach of the viewed stations. There are on average 7,154 distinct namedprograms and 54,086 30 minute airing timeslots each day not includinglocal broadcasts.

The processing described above may avoid illusory view events. Afterjoining the guide data 264 to the set-top-box data 262, the data sourcecombiner 278 creates a set of set-top box viewing events as thefollowing tuple: STB View=(DeviceID, StationEvent, Program, DateTime,TimeZone). StationEvent indicates which station is associated with aviewing event, and program indicates which TV program was aired on thestation at the recorded date and time (based on DateTime and TimeZone).The DeviceID can be linked to the billing name and address for theperson who subscribes to the set-top box through an anonymizationprocess described below.

When the customer purchases a product, the advertiser obtainsinformation about the person's name and address as part of a credit cardtransaction. This information can be used to create conversion data formodeling the impact of TV advertising. Advertiser conversion data caninclude purchase value information, segments assigned by the Advertiser,acquisition and chum date. Acquisition date may be compared againstviewing events, and the system may sum the ad exposures that occurredprior to the conversion. Conversion data contains the following salientfields: Conversion=(CustomerName, Address, DateTime, TimeZone).

The billing name and address (or an anonymized identifier representingthe billing name and/or address) from the set-top box data may bematched against CustomerName and/or address from the sales data (or ananonymized identifier representing the purchaser name and/or address) toconnect viewing events to sales events (conversions).

The system next identifies whether the set-top box viewer saw ads for aparticular advertisement campaign. Ad occurrence data is recorded byseveral media tracking services including SQAD and BVS. SQAD collectstheir data from stations, where-as BVS embeds digital watermarks intothe ads which are detectible in the video stream as it is received bytelevision sets around the country. Based on ad occurrence data, thereare approximately 19 million TV ad airings per month in the UnitedStates, which span the entirety of national, local, broadcast and cablenetworks. For conversion tracking purposes, the system filters the adoccurrence data to the one advertiser for whom conversions are beingtracked. The system then has a tuple which indicates the times that thead was aired. The tuple is as follows; AdOcc=(Station, Program,DateTime, TimeZone). These tuples may be matched against station,program, time and time zone fields in the combined set-top box data andguide data to identify which viewing events resulted in ad impressions.A timing between the sales event and viewing events may then be used toattribute a particular conversion event to one or more ad impressions.

After the combined records are generated that match viewing events,purchase events and ad events, the data may be loaded into one or moredata stores 276 (e.g., databases) for use with any upstream mediasystems. These combined records may be used to support media planningthrough purchase suggestions, revenue predictions, pricing suggestions,performance results, etc. Additionally, an analytics engine 274 of theadvertisement platform 215 may use the data to set up experiments,perform real time tracking of an advertisement campaign, optimize anadvertisement campaign in real time, determine a landscape for anadvertisement campaign, and so forth.

In one embodiment an objective is to join each of the four data sourcestogether and then to start counting ad exposures on each viewer andidentify the probability of conversion as a function of weight andtargeting. Analytics engine 274 uses the combined records to determineprobability of conversion as a function of ad weight and degree oftargetedness.

We will define Ad Weight A as 1000*the number of ads viewed by anindividual per week prior to converting. Ad weight may be computed asfollows:

${A\left( {i,\alpha,\tau} \right)} = {\frac{1000 \cdot {{imp}\left( {i,\alpha} \right)}}{{{weeks}(i)} \cdot {{TVHH}(i)}} = \frac{{1000 \cdot \Sigma_{m:{{{v{({i,m})}}{{\Lambda occ}{({a,m})}}{\Lambda a}} \in \alpha}}}1}{\left( \text{1/7} \right) \cdot {\Delta\left\lbrack {{{\min t}(m)},\left\{ \begin{matrix}{{T(i)},} & {{if}\mspace{14mu}{c(i)}} \\{{{\max t}(m)},} & {otherwise}\end{matrix} \right\rbrack} \right.}}}$∀a ∈ α, ∀m:v(i, m)Λ . . . t(m) ≥ min t(n)Λt(m) ≤ max t(n) : ∀occ(a, n)Λ . . . t(m) ≤ τΛt(m) ≥ τ − d/2

where a are a set of ads for an advertiser, v(i,m) is true if person iviewed media instance m, occ(a,m) is true if an advertisement a aired onmedia instance m. t(m) is the date that media instance m aired. c(i) istrue if person i converted. T(i) is the date person i converted, andΔ(t₁, t₂) returns the days between dates t₁ and t₂. The denominatorrepresents the number of weeks during which the viewer was able to viewads. This is limited to the set of dates when the ad was running, andwhen the person had an opportunity to view it.

A match to determine if a viewer both saw an ad v(i,m), at the same timeas the ad aired occ(a,m) is the most computationally expensive part ofthe above join. It is possible to join where the media instance are onesecond viewing buckets—and so a match is achieved if the ad airs and theuser views the media at the same second. The match can be sped up byincreasing the size of the matching bucket. In addition, there are oftensome clock differences between MSOs and set-top box hardware units, andso using a larger time window (bucket) also helps to improve thereliability of the match. The analytics engine 274 can define a matchingtime window (bucket) of size B in seconds. The analytics engine 274truncates the time stamp for the ad occurrences as well as for theviewing records with the same window or bucket. We then define a matchas

If viewminutes(i,m)≥B*b then v(i,m)=true

where b is a percentage of the window that needs to be viewed. In theexamples that follow, a window size is equal to 15 minutes with b=0.7.This style of matching is not as accurate, and may incorrectly return anad view when the user had actually channel-switched. Therefore, in oneembodiment the AdWeight is an upper bound on the true AdWeight. Howeverit creates up to a 900× speedup which is valuable in practice. Afterusing this technique, the ad weight calculated for the landscape may bereduced as follows:

A(i,a)′=A(i,a)·b

Targeting is challenging on TV because ads are not routed toindividuals. Instead, ads are placed on specific programs, rotations,times of day. etc. In order to do this, a definition of targeting isused herein which is compatible with TV systems. Targetedness, or degreeof targetedness, is the percentage of viewers who are like theconverting customer. The degree of targetedness is equal to “probabilityof buyer”. Accordingly, the higher a targetedness rating for a viewer,the greater the probability that the viewer will convert. We provide twotargetedness metrics: (a) Direct Targeting and (b) DemographicTargeting.

Direct Targeting looks at what known buyers (converters) of the productare watching, and then creates a probability for each media instance.The method calculates the probability of buyer given the TV programmingmix that the individual who is being scored is watching. In other words,the analytics engine reviews all programs viewed, sums up the buyers inthat pool and divides by the number of viewers, as follows:

${r\left( {i,\alpha,\tau} \right)} = \frac{\Sigma_{m:{v{({i,m})}}}\Sigma_{j \neq i}{\text{1:}\text{c}}(j){{\Lambda v}\left( {j,m} \right)}}{\Sigma_{m:{v{({i,m})}}}\Sigma_{k \neq i}{\text{1:}\text{v}}\left( {k,m} \right)}$

where i is an individual set-top box viewer who is being scored, m isone of the media instances which viewer i has watched. B(m) is thenumber of buyers viewing media program m, and V(m) is the universe ofall viewers of media m. For example, if there were 10 buyers out of 100on Program A, and 1 out of 100 on Program B, and an individual viewedonly Program A and B, then their buyer probability is 5.5%.

Direct buyer probability calculation can run into difficulty when thereare few conversions. Another method is to use the demographics of mediato calculate the probability of a conversion from this media. Using thismethod, analytics engine 274 decomposes each individual set-top boxviewer into a multiple element demographic variable-value vector I,which in one embodiment is a 400 element vector. Analytics engine 274then compares the viewer demographics to the demographics of purchasersof the advertiser's product P. This method has the advantage that itwill work across all possible TV programs, regardless of the potentialsparsity of buyers in the population. Demographic targeting can becomputed according to the following equation:

${r\left( {i,\alpha,\tau} \right)} = \frac{P \cdot I}{{P} \cdot {I}}$

The function for estimated conversions due to TV, for any individual ithen becomes:

E[Conv(i, a, τ)|A(i, a, τ), r(i, a, τ), τ]=A(i, a, τ)·ƒ(r(i, a, τ),τ)=Z(t,a, τ)·[C _(r) ·r(i, a, τ)+O _(τ)]

Where A(i, a, τ) is the Ad Weight in impressions per Million Householdsper week (Imp/MHH/Wk), and ƒ is a Targeting function which maps r(i, a,τ) to expected conversions per impression, and which can be estimated byleast squares regression of r against observed conversion rate perimpression, O_(τ) are conversions occurring for untargeted impressions,and C_(z) are the conversions per targeted impression. The conversionper targeted impression function is also able to vary with thetime-period being observed, and analytics engine 274 can use this tocreate a time series of conversions due to television.

In an example, the demographics targeting method was applied for a largebrick and mortar retailer in United States to compute targetedness. Thecompany tracked approximately 120,000 unique customer sales per month.These converting customers were joined to STB viewers using thethree-way anonymization method to ensure that identities of viewers andpurchasers were not disclosed in any way.

Over an 8 month period, there were 609 positive instances of STB viewerswho converted. The retailer had run approximately 643 Imp/MHH/Wk for 8months. Based on this, 16.19% of the STB viewers had been watching TV atexactly the time that the retailer ad played and had seen the ad atleast once during the 34 weeks that were monitored. Only 0.26% ofviewers saw the ad once per week. FIG. 5 illustrates a percentage ofset-top box viewers who saw 1, 2, 3 or more ads.

Conversion rate increases were observed with the number of times thatthe ad was seen (e.g., with increased ad weight). Without seeing the adthe conversion rate is only 0.048%. If the ad is seen once per week thenthe conversion rate increases by a factor of 7.5 to 0.36%.

Advertisement weight verses conversion probability is shown in FIG. 6.As shown, increases in the number of impressions per MHH per week causecorresponding increases in conversion probability. FIG. 7 illustratesconversion probability as a function of advertising weight expressed astotal number of impressions. As the total number of impressions (views)of an ad increase, the probability that a viewer will purchase theadvertised product or service increases, as shown.

Conversion rate also increases with targeting (also referred to astargetedness), which is represented mathematically by a Tratio value.FIG. 8 shows the relationship between demographic match and probabilityof conversion. As the degree of targetedness increases, the probabilitythat a viewer of the ad will purchase the product or service that isadvertised increases, as shown. FIG. 9 is another graph showing theprobability of conversion after a single advertisement impression basedon degree of targetedness.

Combining ad weight and targeting, the graph shown in FIG. 10 isproduced. This buckets the data into discrete combinations of ad weightand targeting. It can be seen clearly that high ad weight, high targetedmedia produces a high conversion rate. Fitting this data using the abovelinear function, the conversion function from television ad weight givena degree of targeting as measured by r(i, a, τ) can be estimated as:

E[Conv(i, a, τ)]=A(i, a, τ)·[0.000037*r(i, a, τ)+0.000029]

Based on the ad spending of 643 Imp/MHH/Wk recorded over the 8 months,and targetedness of r=0.0778, this meant that(643/1000)·1500000·0.000037·0.0778=2,236 conversions per week were dueto TV, 5,289 conversions occurred per week, meaning that TVadvertisements had lifted sales by 88% over this period.

Analytics engine 274 can identify the value of TV targeting and adweight, and use that to identify conversions due to television over anyperiod. The above function is based on empirically collected data.Analytics engine 274 can generate a similar function, but which is localin time. The similar function effectively models different behaviorduring different times of the year. A spline version of this function isbelow where G is a radial basis function, z are centroids for time, andC_(z) maps targetedness to conversions per impression, where:

${E\left\lbrack {{Conv}\left( {i,\alpha,\tau} \right)} \right\rbrack} = {{A\left( {i,\alpha,\tau} \right)} \cdot {\sum\limits_{z}^{\;}\;{C_{z} \cdot {G\left( {\tau - t_{z}} \right)} \cdot {r\left( {i,\alpha,\tau} \right)}}}}$

In one embodiment, analytics engine 274 determines counts of previous adexposures for purchasers and/or non-purchasers of the advertised productor service. Analytics engine 274 may also determine buyer exposure toads, and report on such buyer exposure. For example, analytics engine274 may use the combined data to compute the total number of buyers whowere exposed to ads. Such exposures may be reported out by time. Allpurchasers may be used as a set of total converters, and may be comparedagainst the number of converters who also were exposed to theadvertisement, campaign to determine whether the ads are being deliveredto the right audience. The ratio of purchasers who viewed the ads tototal purchasers may be used as a quality metric to measure theappropriateness of the selected programs for the ad campaign.

The platform consumers 205 may include an agency 230 (e.g., anadvertising agency) and an advertiser 232 (e.g., a manufacturer of aproduct or service who wishes to advertise that product or service).Results of the real time tracking, advertisement optimizationsuggestions, advertising models, etc. may be provided to the platformconsumers 205 to enable them to fully understand and optimize theiradvertisement campaigns in real time or pseudo-real time (e.g., whilethe campaigns are ongoing).

In one embodiment, audience data 220 and sales data 225 are anonymizedby an anonymization service before the data is provided to advertisementplatform 215. The anonymization service may extract personallyidentifiable information (PII) from the data feeds and route the PII toa separate pipeline for secure storage. The anonymization service mayadditionally add unique identifiers to each article of data.

In another embodiment, the system uses a pool of buyers—people who havepreviously bought the product or used the service—and then provides fortracking of how well the ad campaign is doing at hitting those buyers,or others that are predicted to be high-probability buyers. This is doneby measuring the number of ad exposures of persons in the buying set,and counting up those exposures each day. It is then possible to reporton several metrics to show how well the campaign is performing including(a) how many buyer exposures per day were generated, (b) how manydistinct buyers were delivered the ad per day, (c) what was the cost perbuyer reached, and (d) how many buyers per million impressions (BPM)were reached. The latter metric of buyers per million impressions or BPMis a very useful metric to show the degree of targeting in the adcampaign. These metrics can also be reported by program, stationday-hour, and other units of purchase, and so show whether mediapurchases were effective in getting the ad in front of buyers.

FIG. 11 is a flow diagram illustrating a method 1100 of tracking andmanaging an advertisement campaign based on set-top box data, accordingto one embodiment. The method 1100 may be performed by processing logicthat comprises hardware (e.g., processor, circuitry, dedicated logic,programmable logic, microcode, etc.), software (e.g., instructions runon a processor to perform hardware simulation), or a combinationthereof. The processing logic is configured to track and manage anadvertisement campaign such as a national advertisement campaign. In oneembodiment, method 1100 may be performed by a processor, as shown inFIG. 13. In one embodiment, method 1100 is performed by an advertisementplatform (e.g., by analytics engine 274 and data source combiner 278 ofadvertising platform 215 discussed with reference to FIG. 2).

Referring to FIG. 11, at block 1105 of method 1100 processing logicreceives set-top box data, advertisement data, sales data andprogramming guide data. The set-top box data and sales data may havebeen anonymized by a third party anonymization service as describedabove herein. Accordingly, the received set-top box data and sales datamay not include any personally identifying information (PII). At block1110, processing logic associates set-top boxes with viewers. Processinglogic may make these associations based on billing and/or customerrecords of set top box users collected from service providers thatprovide services to the set-top boxes (e.g., from cable televisionproviders, satellite, television providers, and so on). Set-top box datamay be associated with a viewer who has signed up for the service withthe service provider. This viewer may be representative of a householdin one embodiment. Additional data on households may be received fromthird party services such as Axciom. The additional household data mayidentify multiple viewers who live in a household, wherein one of theviewers is the viewer associated with a user account (e.g., who pays thebills for the television service). Subsets of the set-top box dataassociated with a household may be associated with different viewers ofthe household.

At block 1112, processing logic filters the set-top box data todetermine viewing events. The set-top box data may be received as aseries of set-top box events, which may include power on events, poweroff events, tune events (tuning to a specific channel), and numerousother types of events. Processing logic may filter out all events otherthan tune events and power on/off events. Processing logic may then usethe tune events and power on/off events to determine viewing events andto generate viewing event records, as set forth above. A viewing eventmay have a specified time window. For example, in one embodiment, 15minute viewing events are generated. Alternatively, 30 minute viewingevents, hour viewing events, and so forth may be generated. Each viewingevent may be associated with one or more viewers of a household.Processing logic may update the viewing event records by addinginformation from the received programming guide data. Processing logicdetermines what programs are associated with each viewing event, andadds the program data to the viewing event record.

At block 113, processing logic calculates advertisement exposure eventsbased on combining the viewing events with the advertisement data.Processing logic further combines the advertisement data with theviewing event records by determining whether ads were aired during aprogram associated with a viewing event record and/or how many ads wereaired during the program. The number of aired ads may then be added tothe viewing event record. Thus, each viewing event record includes an adweight for that viewing event record. At block 1115, the viewing eventrecords for an individual viewer are combined to determine the ad weightfor that viewer. Additionally, viewing event records for all viewers maybe combined to determine a total average ad weight applied to viewersand/or other statistical information regarding ad weight (e.g., averagead weight of all viewers for a particular program).

In one embodiment, processing logic may determine that a viewer viewed aportion of a program, and may assign a determined program leveladvertising weight for the portion of the program. For example, if an adis shown 4 times in a program, then the program may have an ad weight of4. If a viewer viewed ¼ the program, then the viewing event for theviewer of the program may be assigned an ad weight of 1. In oneembodiment, a time window is set for each viewing event (e.g., 15minutes). An ad exposure event may be determined if a viewer viewed aprogram for at least a threshold amount of the time window (e.g., 10% ofthe time window).

At block 1120, processing logic combines the set-top box data with thesales data to determine a degree of targetedness for each individualviewer. In one embodiment, the sales data is matched against the viewingevent records based on name and address or based on a unique identifierassociated with a household or viewer. Any of the aforementionedtechniques for determining degree of targetedness may be used. Forexample, processing logic may identify ail programs viewed by a viewer,determine a first quantity of viewers that viewed a program, determine asecond quantity viewers that also purchased the product or service, anddivide the second quantity by the first quantity to determine the degreeof targetedness for the viewer. Similar measurements may also be made atthe household level. Alternatively, processing logic may generate afirst multi-level vector based on demographic information for theviewer, generate a second, multi dimensional vector for the buyers ofthe product or service, and determine a distance between the firstmulti-dimensional vector and the second multi-dimensional vector.

At block 1125, processing logic generates a multi-dimensional model ofadvertising effectiveness that models combined effects of advertisingweight and degree of targetedness. The multi-dimensional model may bedeveloped using the computed degrees of targetedness and ad weights ofthe individual viewers in combination with the sales data. These valuesmay be used along with received data to determine an effect that the adweight has on sales metrics and to determine an effect that the degreeof targetedness has on the sales metrics. At block 1130, processinglogic applies the multi-dimensional model to the received set-top boxdata and the received sales data to track a number of sales that areattributable to the advertisement campaign. At block 1135, processinglogic uses the multi-dimensional model to predict a number ofconversions for a particular advertisement weight applied to a programhaving a particular degree of targetedness. For example, viewers of aparticular program may have a demographic similarity to the purchasersof the product or service, represented by the degree of targetedness orTratio. The multi-dimensional model may be used to determine thepredicted number of conversions that will occur if 1 ad per week isshown, if 4 ads per week are shown, and so on. This data may be used tomanage the advertisement campaign.

At block 1140, processing logic determines whether new data (e.g., newset-top box data and/or new sales data) has been received. If new datais received, the method continues to block 1145 and themulti-dimensional model is updated using the new data. This may includefirst performing the operations of blocks 1110-1120 for the new data.Once the multi-dimensional model is updated, the method returns to block1130. If no new data is received, the method ends.

FIG. 12 is a flow diagram illustrating a method 1200 of tracking andmanaging an advertisement campaign based on set-top box data, accordingto one embodiment. The method 1200 may be performed by processing logicthat comprises hardware (e.g., processor, circuitry, dedicated logic,programmable logic, microcode, etc.). software (e.g., instructions runon a processor to perform hardware simulation), or a combinationthereof. The processing logic is configured to track and manage anadvertisement campaign such as a national advertisement campaign, in oneembodiment, method 1200 may be performed by a processor, as shown inFIG. 13. In one embodiment, method 1200 is performed by an advertisementplatform (e.g., by analytics engine 274 and data source combiner 278 ofadvertising platform 215 discussed with reference to FIG. 2).

Referring to FIG. 12, at block 1205 of method 1200 processing logicreceives set-top box data, advertisement data, sales data andprogramming guide data. The set-top box data and sales data may havebeen anonymized by a third party anonymization service as describedabove herein. Accordingly, the received set-top box data and sales datamay not include any personally identifying information (PII).

At block 1210, processing logic filters the set-top box data todetermine viewing events. At block 1220, processing logic combines theset-top box data with the sales data and the advertisement data todetermine a total number of purchasers of the product or service whowere exposed to the advertisement campaign. At block 1225, processinglogic determines timings of the purchases made by the purchasers whowere exposed to the advertisement campaign. At block 1230, processinglogic reports the timings of when purchasers made their purchases inrelation to when the ads were viewed. Accordingly, processing logic maydetermine how many days typically elapsed between viewing of an ad forthe product or service and purchase of the product or service.Processing logic may also determine the number of airings that occurredbefore the product or service was purchased, and other information.Additionally, processing logic may report on timings of when products orservices were purchased without indicating an amount of time thatelapsed since a most recent ad exposure.

FIG. 13 illustrates a diagrammatic representation of a machine in theexemplary form of a computer system 1300 within which a set ofinstructions, for causing the machine to perform any one or more of themethodologies discussed herein, may be executed. The system 1300 may bein the form of a computer system within which a set of instructions, forcausing the machine to perform any one or more of the methodologiesdiscussed herein, may be executed. In alternative embodiments, themachine may be connected (e.g., networked) to other machines in a LAN,an intranet, an extranet, or the Internet. The machine may operate inthe capacity of a server machine in client-server network environment.The machine may be a personal computer (PC), a set top box (STB), aserver, a network router, switch or bridge, or any machine capable ofexecuting a set of instructions (sequential or otherwise) that specifyactions to be taken by that machine. Further, while only a singlemachine is illustrated, the term “machine” shall also be taken toinclude any collection of machines that individually or jointly executea set (or multiple sets) of instructions to perform any one or more ofthe methodologies discussed herein.

The exemplary computer system 1300 includes a processing device (e.g.,one or more processors) 1302, a main memory 1304 (e.g., read only memory(ROM), flash memory, dynamic random access memory (DRAM) such assynchronous DRAM (SDRAM)), a static memory 1306 (e.g., flash memory,static random access memory (SRAM)), and a data storage device 1318,which communicate with each other via a bus 1330.

Processing device 1302 represents one or more general-purpose processorssuch as a microprocessor, central processing unit, or the like. Moreparticularly, the processing device 1302 may be a complex instructionset computing (CISC) microprocessor, reduced instruction set computing(RISC) microprocessor, very long instruction word (VLIW) microprocessor,or a processor implementing other instruction sets or processorsimplementing a combination of instruction sets. The processing device1302 may also be one or more special-purpose processing devices such asan application specific integrated circuit (ASIC), a field programmablegate array (FPGA), a digital signal processor (DSP), network processor,or the like. The processing device 1302 is configured to execute theinstructions 1326 for performing the operations and steps discussedherein.

The computer system 1300 may further include a network interface device1308 which may communicate with a network 1320. The computer system 1300also may include a video display unit 1310 (e.g., a liquid crystaldisplay (LCD) or a cathode ray tube (CRT)), an alphanumeric input device1312 (e.g., a keyboard), a cursor control device 1314 (e.g., a mouse, atouch screen, a touch pad, a stylus, etc.), and a signal generationdevice 1316 (e.g., a speaker).

The data storage device 1318 may include a computer-readable storagemedium 1328 on which is stored one or more sets of instructions (e.g.,instructions 1326 for an analytics engine 1390) embodying any one ormore of the methodologies or functions described herein. Theinstructions 1326 may also reside, completely or at least partially,within the main memory 1304 and/or within the processing device 1302during execution thereof by the computer system 1300, the main memory1304 and the processing device 1302 also constituting computer-readablemedia. The instructions may further be transmitted or received over anetwork 1320 via the network interface device 1308.

While the computer-readable storage medium 1328 is shown in an exemplaryembodiment to be a single medium, the term “computer-readable storagemedium” should be taken to include a single medium or multiple media(e.g., a centralized or distributed database, and/or associated cachesand servers) that store the one or more sets of instructions. The term“computer-readable storage medium” shall also be taken to include anymedium that is capable of storing, encoding or carrying a set ofinstructions for execution by the machine and that cause the machine toperform any one or more of the methodologies of the present invention.The term “computer-readable storage medium” shall accordingly be takento include, but not be limited to, solid-state memories, optical media,and magnetic media.

In the above description, numerous details are set forth. It will beapparent, however, to one of ordinary skill in the art having thebenefit of this disclosure, that embodiments of the invention may bepracticed without these specific details. In some instances, well-knownstructures and devices are shown in block diagram form, rather than indetail in order to avoid obscuring the description.

Some portions of the detailed description are presented in terms ofalgorithms and symbolic representations of operations on data bitswithin a computer memory. These algorithmic descriptions andrepresentations are the means used by those skilled in the dataprocessing arts to most effectively convey the substance of their workto others skilled in the art. An algorithm is here, and generally,conceived to be a self-consistent sequence of steps leading to a desiredresult. The steps are those requiring physical manipulations of physicalquantities. Usually, though not necessarily, these quantities take theform of electrical or magnetic signals capable of being stored,transferred, combined, compared, and otherwise manipulated. It hasproven convenient at times, principally for reasons of common usage, torefer to these signals as bits, values, elements, symbols, characters,terms, numbers, or the like.

It should be borne in mind, however, that all of these and similar termsare to be associated with the appropriate physical quantities and aremerely convenient labels applied to these quantities. Unlessspecifically stated otherwise as apparent from the above discussion, itis appreciated that throughout the description, discussions utilizingterms such as “receiving,” “generating,” “determining,” “calculating,”“introducing,” “providing,” “selecting,” “updating,” “adjusting,”“modifying,” “computing,” “using,” “applying,” “comparing,” “analyzing,”“tracking,” “incorporating,” “combining,” “predicting,” “performing,”“reducing,” “increasing,” “making,” “monitoring,” “maintaining,”“updating,” “testing,” “measuring,” “identifying,” or the like, refer tothe actions and processes of a computer system, or similar electroniccomputing device, that manipulates and transforms data represented asphysical (e.g., electronic) quantities within the computer system'sregisters and memories into other data similarly represented as physicalquantities within the computer system memories or registers or othersuch information storage, transmission or display devices.

Embodiments of the invention also relate to an apparatus for performingthe operations herein. This apparatus may be specially constructed forthe required purposes, or it may comprise a general purpose computerselectively activated or reconfigured by a computer program stored inthe computer. Such a computer program may be stored in a computerreadable storage medium, such as, but not limited to. any type of diskincluding floppy disks, optical disks, CD-ROMs, and magnetic-opticaldisks, read-only memories (ROMs), random access memories (RAMs), EPROMs,EEPROMs, magnetic or optical media, or any type of media suitable forstoring electronic instructions.

The algorithms and displays presented herein are not inherently relatedto any particular computer or other apparatus. Various general purposesystems may be used with programs in accordance with the teachingsherein, or it may prove convenient to construct a more specializedapparatus to perform the required method steps. The required structurefor a variety of these systems will appear from the description below.In addition, the present invention is not described with reference toany particular programming language. It will be appreciated that avariety of programming languages may be used to implement the teachingsof the invention as described herein.

It is to be understood that the above description is intended to beillustrative, and not restrictive. Many other embodiments will beapparent to those of skill in the art upon reading and understanding theabove description. The scope of the invention should, therefore, bedetermined with reference to the appended claims, along with the fullscope of equivalents to which such claims are entitled.

1-22. (canceled)
 23. A method comprising: setting a time windowassociated with a portion of each electronic content instance among aplurality of electronic content instances on which an advertisementcampaign for a product or service was presented based on a predeterminedtime window size, the time window size being greater than a duration ofan advertisement of the advertisement campaign; determining whether anindividual viewer of a plurality of viewers viewed the electroniccontent instance for at least a threshold percentage of the time window;upon determining that the individual viewer viewed the electroniccontent instance for at least the threshold percentage of the timewindow, identifying a single advertising exposure event for theindividual viewer; calculating, for each viewer of the plurality ofviewers, an advertising weight as a function of a number of exposures ofthe advertisement as applied to the viewer based on the advertisingexposure events; calculating, for each viewer, a score representing adegree-of-targetedness between a target for the advertisement campaignand the viewer; generating a multi-dimensional model of advertisingeffectiveness that models combined effects of the advertising weight andthe degree of targetedness based on a combination of computedadvertising weights and degrees of targetedness for the plurality ofviewers; and applying the multi-dimensional model to electronic contentdevice data a plurality of electronic content devices associated withthe plurality of viewers and sales data identifying a plurality ofpurchasers of the product or service to track a number of salesattributable to the advertisement campaign.
 24. The method of claim 23,wherein associating the plurality of electronic content devices with theplurality of viewers comprises: associating each electronic contentdevice of the plurality of electronic content devices with a householdof a plurality of households based on billing or subscription records ofa person who is paying for a television service provided by one of theplurality of electronic content devices; and associating each householdof the plurality of households with additional persons in the householdbased on third party data about the additional persons residing withinthe household.
 25. The method of claim 23, wherein the degree oftargetedness is further calculated by measuring a probability that anindividual exposed to the advertisement campaign will purchase theproduct or service, and wherein determining the degree of targetednessas applied to an individual viewer comprises: identifying programsviewed by the individual viewer; determining a first quantity ofindividual viewers that viewed the programs; determining a secondquantity of the individual viewers that also purchased the product orservice; and dividing the second quantity by the first quantity.
 26. Themethod of claim 23, further comprising: determining a total number ofpurchasers of the product or service; measuring a number of thepurchasers who viewed at least one advertisement of the advertisementcampaign; and reporting the number of the purchasers who viewed at leastone advertisement based on time.
 27. The method of claim 23, wherein thedegree-of-targetedness is calculated by generating a firstmulti-dimensional vector based on consumer enrichment data for theindividual viewer, generating a second multi-dimensional vector based onconsumer enrichment data for buyers of the product or service, andcalculating a difference between the first multi-dimensional vector andthe second multi-dimensional vector.
 28. The method of claim 23, furthercomprising: managing the advertisement campaign based on the advertisingweight and the degree-of-targetedness.
 29. The method of claim 23,wherein the score representing the degree-of-targetedness is a pluralityof values including a direct targeting metric and a demographictargeting metric, and the demographic targeting metric is based on aprobability of a conversion for the product or service from a mediainstance on which the advertisement campaign was presented.
 30. Atangible, non-transitory computer readable storage medium havinginstructions that, when executed by a processing device, cause theprocessing device to perform operations comprising: setting a timewindow associated with a portion of each electronic content instanceamong a plurality of electronic content instances on which anadvertisement campaign for a product or service was presented based on apredetermined time window size, the time window size being greater thana duration of an advertisement of the advertisement campaign;determining whether an individual viewer of a plurality of viewersviewed the electronic content instance for at least a thresholdpercentage of the time window; upon determining that the individualviewer viewed the electronic content instance for at least the thresholdpercentage of the time window, identifying a single advertising exposureevent for the individual viewer; calculating, for each viewer of theplurality of viewers, an advertising weight as a function of a number ofexposures of the advertisement as applied to the viewer based on theadvertising exposure events; calculating, for each viewer, a scorerepresenting a degree-of-targetedness between the advertisement campaignand the viewer; generating a multi-dimensional model of advertisingeffectiveness that models combined effects of the advertising weight andthe degree of targetedness based on a combination of computedadvertising weights and degrees of targetedness for the plurality ofviewers; and applying the multi-dimensional model to electronic contentdevice data a plurality of electronic content devices associated withthe plurality of viewers and sales data identifying a plurality ofpurchasers of the product or service to track a number of salesattributable to the advertisement campaign.
 31. The tangible,non-transitory computer readable storage medium of claim 30, whereinassociating the plurality of electronic content devices with theplurality of viewers comprises: associating each electronic contentdevice of the plurality of electronic content devices with a householdof a plurality of households based on billing or subscription records ofa person who is paying for a television service provided by one of theplurality of electronic content devices; and and associating eachhousehold of the plurality of households with additional persons in thehousehold based on third party data about the additional personsresiding within the household.
 32. The tangible, non-transitory computerreadable storage medium of claim 30, wherein generating themulti-dimensional model comprises: determining an effect that theadvertising weight of the advertisement has on sales metrics identifiedfrom the sales data; and determining an effect that the degree oftargetedness for the advertisement has on the sales metrics.
 33. Thetangible, non-transitory computer readable storage medium of claim 30,the operations further comprising: using the multi-dimensional model topredict a number of conversions for a particular advertisement weightapplied to a program having a particular degree of targetedness.
 34. Thetangible, non-transitory computer readable storage medium of claim 30,wherein determining an advertising weight as applied to an individualviewer comprises: receiving programming guide data; analyzing theelectronic content device data and combining with the programming guidedata to determine programs watched by the individual viewer over a timeperiod; calculating the advertising exposure events for the individualviewer by further combining the advertising data to the combined viewingdata and programming data; and aggregating the advertising exposureevents for the individual viewer.
 35. The tangible, non-transitorycomputer readable storage medium of claim 30, wherein the degree oftargetedness is further calculated at least in part by measuring aprobability that an individual exposed to the advertisement campaignwill purchase the product or service, and wherein determining the degreeof targetedness as applied to an individual viewer comprises:identifying programs viewed by the individual viewer; determining afirst quantity of individual viewers that viewed the programs;determining a second quantity of the individual viewers that alsopurchased the product or service; and dividing the second quantity bythe first quantity.
 36. The tangible, non-transitory computer readablestorage medium of claim 30, the operations further comprising:determining a total number of purchasers of the product or service;measuring a number of the purchasers who viewed at least oneadvertisement of the advertisement campaign; and reporting the number ofthe purchasers who viewed at least one advertisement based on time. 37.The tangible, non-transitory computer readable storage medium of claim30, wherein the degree-of-targetedness is calculated by generating afirst multi-dimensional vector based on consumer enrichment data for theindividual viewer, generating a second multi-dimensional vector based onconsumer enrichment data for buyers of the product or service, andcalculating a difference between the first multi-dimensional vector andthe second multi-dimensional vector.
 38. The tangible, non-transitorycomputer readable storage medium of claim 30, the operations furthercomprising: managing the advertisement campaign based on the advertisingweight and the degree-of-targetedness.
 39. A computing devicecomprising: a memory; and a processing device coupled to the memory, theprocessing device to: set a time window associated with a portion ofeach electronic content instance among a plurality of electronic contentinstances on which an advertisement campaign for a product or servicewas presented based on a predetermined time window size, the time windowsize being greater than a duration of an advertisement of theadvertisement campaign; determine whether an individual viewer of aplurality of viewers viewed the electronic content instance for at leasta threshold percentage of the time window; upon determining that theindividual viewer viewed the electronic content instance for at leastthe threshold percentage of the time window, identify a singleadvertising exposure event for the individual viewer; calculate, foreach viewer of the plurality of viewers, an advertising weight as afunction of a number of exposures of the advertisement as applied to theviewer based on the advertising exposure events; calculate, for eachviewer, a score representing a degree-of-targetedness between theadvertisement campaign and the viewer; generate a multi-dimensionalmodel of advertising effectiveness that models combined effects of theadvertising weight and the degree of targetedness based on a combinationof computed advertising weights and degrees of targetedness for theplurality of viewers; and apply the multi-dimensional model toelectronic content device data a plurality of electronic content devicesassociated with the plurality of viewers and sales data to track anumber of sales attributable to the advertisement campaign.
 40. Thecomputing device of claim 39, wherein the degree-of-targetedness iscalculated by generating a first multi-dimensional vector based onconsumer enrichment data for the individual viewer, generating a secondmulti-dimensional vector based on consumer enrichment data for buyers ofthe product or service, and calculating a difference between the firstmulti-dimensional vector and the second multi-dimensional vector. 41.The computing device of claim 40, the computing device furtherconfigured to: manage the advertisement campaign based on theadvertising weight and the degree-of-targetedness.
 42. The computingdevice of claim 40, wherein the score representing thedegree-of-targetedness is a plurality of values including a directtargeting metric and a demographic targeting metric, and the demographictargeting metric is based on a probability of a conversion for theproduct or service from a media instance on which the advertisementcampaign was presented.